AIP Sperimentale 11 - 13 September 2025 | Torino
Overlapping index (\(\eta\))
Common Language Effect Size (CLES)
The overlapping (\(\eta\)) is an index of effect size and varies from 0 and 1.
It is \(0\) when the two distributions are completely disjoint and \(1\) when there is complete overlap.
It express the probability that an observation randomly extracted from one group will be higher than the one extracted from the other group.
n = 10, 50, 100, 300, 500, equal in the two samples.
Mean difference: \(\delta\) = 0, 1, 2;
Variability: \(\sigma\) = 1, 3, 5;
Skewness: \(\alpha\) =0, 5;
For the total combination of \(5 × 3 × 3 × 2 = 90\) conditions we generated \(500\) sets of data on which we performed the analysis.
We compared the two effect sizes on the following indexes of bias:
The Absolute mean bias (AMB)
The Relative mean bias (RMB)
The Mean Square Error (MSE)
The Overlapping index presents less Relative Mean Bias (RMB) compared to the CLES in conditions with no homogeneity of variance between the two groups.
Next Steps
We will compare the Overlapping Index to other Effect Sizes but the problem of differences in scale needs to be solved.
Suggestions?
We compared the two effect sizes on the following indexes of bias:
The Absolute mean bias (AMB): obtained as the mean of the absolute bias: \(1/B\sum_b (\hat{\theta}_{bj}-\theta_j)\) in which \(\theta_j\) is the value in the condition \(j\) and \(\hat{\theta}_{bj}\) the one estimated in the replication \(b\) in the condition \(j\).
The Relative mean bias (RMB): obtained as the mean of relative bias: \(1/B\sum_b (\hat{\theta}_{bj}-\theta_j)/\theta_j\). Relative bias included between -0.1 and 0.1 are considered acceptable.
The Mean Square Error (MSE), defined as \(\sigma_j^2 + [1/B \sum_b (\hat{\theta}_{bj}-\theta_j)]^2\), with \(\sigma_j^2\) being the variance of the estimates across replications. The lower the MSE, the closer is forecast to actual.

ambra.perugini@phd.unipd.it